This project attempts to fulfill the assignment outlined in https://www.gastrograph.com/blogs/interviewing-data-science-interns.
“Using only partial least squares regression for generalized linear models (the plsRglm package in R), build an ensemble model to predict the quality score given to each wine from the Vinho Verde region of Portugal (see the data bullet in the Requirements section below to download the datasets). The data consists of chemical tests on wines and an overall assessment score averaged from at least three professional wine tasters. This is interesting data for AFS as the lack of consistent preferences among professional tasters is one of the reasons our company exists.”
“The rubric for assessment is explained in the Selection Criteria section below. The prediction model should be trained and will be tested on both red and white wine after joining the two datasets. We will split the supplied data into an 80% training set and a 20% hold-out validation set before running your training script with a random seed. We will use mean absolute error as a performance metric.”
Several packages needed to be installed in order to use plsRglm. Other packages are loaded for data exploration, tidying, and analysis. The install commands are commented out rather than removed in case they need to be re-installed on another machine.
#install.packages("mvtnorm")
Warning message:
R graphics engine version 14 is not supported by this version of RStudio. The Plots tab will be disabled until a newer version of RStudio is installed.
#install.packages("bipartite")
# Package 'nloptr' must be installed like this:
# Download https://github.com/stevengj/nlopt/archive/v2.7.1.tar.gz
# Extract contents, then enter the directory and type the following command:
# cmake . && make && sudo make install
# next type this command:
# sudo apt install libnlopt-dev
# Then you can do this:
#install.packages("nloptr")
# Package 'nloptr' is a prerequisite for 'car'
#install.packages("car")
# Packages 'mvtnorm', 'bipartite', and 'car' are prerequisites for 'plsRglm'.
#install.packages("plsRglm")
library("plsRglm") # See https://arxiv.org/abs/1810.01005
#install.packages("tidyverse")
library("tidyverse") # for tidying up the data set
Registered S3 methods overwritten by 'dbplyr':
method from
print.tbl_lazy
print.tbl_sql
[30m── [1mAttaching packages[22m ───────────────────────────────────────────── tidyverse 1.3.1 ──[39m
[30m[32m✓[30m [34mggplot2[30m 3.3.5 [32m✓[30m [34mpurrr [30m 0.3.4
[32m✓[30m [34mtibble [30m 3.1.6 [32m✓[30m [34mdplyr [30m 1.0.8
[32m✓[30m [34mtidyr [30m 1.2.0 [32m✓[30m [34mstringr[30m 1.4.0
[32m✓[30m [34mreadr [30m 2.1.2 [32m✓[30m [34mforcats[30m 0.5.1[39m
[30m── [1mConflicts[22m ──────────────────────────────────────────────── tidyverse_conflicts() ──
[31mx[30m [34mdplyr[30m::[32mfilter()[30m masks [34mstats[30m::filter()
[31mx[30m [34mdplyr[30m::[32mlag()[30m masks [34mstats[30m::lag()[39m
#install.packages("ggplot2")
library("ggplot2") # for visualizations
#install.packages("GGally")
library("GGally")
Registered S3 method overwritten by 'GGally':
method from
+.gg ggplot2
#install.packages('plsdof')
library('plsdof') # Necessary for "plsRglm" to work
Loading required package: MASS
Attaching package: ‘MASS’
The following object is masked from ‘package:dplyr’:
select
#install.packages("moments")
library('moments') # For calculating skewness of distributions
#install.packages("Metrics")
library("Metrics") # For calculating MAE
library("stringr") # For regex
The data are stored in two files. One contains information on red wine and the other on white wine. In both cases, values are separated by semicolons. We will read in both files, add a variable describing the color of the wine, and then merge them into one dataframe.
# read csv's
df.red <- read.csv("data/winequality-red.csv", sep=';')
df.white <- read.csv("data/winequality-white.csv", sep=';')
# add categorical color variables to both sets
df.red['color'] <- 'red'
df.white['color'] <- 'white'
# merge datasets
df <- rbind(df.white, df.red)
The goal is to use these input variables: 1 - fixed.acidity 2 - volatile.acidity 3 - citric.acid 4 - residual.sugar 5 - chlorides 6 - free.sulfur.dioxide 7 - total.sulfur.dioxide 8 - density 9 - pH 10 - sulphates 11 - alcohol 13 - color
To predict this output variable: 12 - quality (score between 0 and 10)
Let’s print out a few things to get an idea of what the data look like:
str(df)
'data.frame': 6497 obs. of 13 variables:
$ fixed.acidity : num 7 6.3 8.1 7.2 7.2 8.1 6.2 7 6.3 8.1 ...
$ volatile.acidity : num 0.27 0.3 0.28 0.23 0.23 0.28 0.32 0.27 0.3 0.22 ...
$ citric.acid : num 0.36 0.34 0.4 0.32 0.32 0.4 0.16 0.36 0.34 0.43 ...
$ residual.sugar : num 20.7 1.6 6.9 8.5 8.5 6.9 7 20.7 1.6 1.5 ...
$ chlorides : num 0.045 0.049 0.05 0.058 0.058 0.05 0.045 0.045 0.049 0.044 ...
$ free.sulfur.dioxide : num 45 14 30 47 47 30 30 45 14 28 ...
$ total.sulfur.dioxide: num 170 132 97 186 186 97 136 170 132 129 ...
$ density : num 1.001 0.994 0.995 0.996 0.996 ...
$ pH : num 3 3.3 3.26 3.19 3.19 3.26 3.18 3 3.3 3.22 ...
$ sulphates : num 0.45 0.49 0.44 0.4 0.4 0.44 0.47 0.45 0.49 0.45 ...
$ alcohol : num 8.8 9.5 10.1 9.9 9.9 10.1 9.6 8.8 9.5 11 ...
$ quality : int 6 6 6 6 6 6 6 6 6 6 ...
$ color : chr "white" "white" "white" "white" ...
head(df)
tail(df)
summary(df)
fixed.acidity volatile.acidity citric.acid residual.sugar
Min. : 3.800 Min. :0.0800 Min. :0.0000 Min. : 0.600
1st Qu.: 6.400 1st Qu.:0.2300 1st Qu.:0.2500 1st Qu.: 1.800
Median : 7.000 Median :0.2900 Median :0.3100 Median : 3.000
Mean : 7.215 Mean :0.3397 Mean :0.3186 Mean : 5.443
3rd Qu.: 7.700 3rd Qu.:0.4000 3rd Qu.:0.3900 3rd Qu.: 8.100
Max. :15.900 Max. :1.5800 Max. :1.6600 Max. :65.800
chlorides free.sulfur.dioxide total.sulfur.dioxide density
Min. :0.00900 Min. : 1.00 Min. : 6.0 Min. :0.9871
1st Qu.:0.03800 1st Qu.: 17.00 1st Qu.: 77.0 1st Qu.:0.9923
Median :0.04700 Median : 29.00 Median :118.0 Median :0.9949
Mean :0.05603 Mean : 30.53 Mean :115.7 Mean :0.9947
3rd Qu.:0.06500 3rd Qu.: 41.00 3rd Qu.:156.0 3rd Qu.:0.9970
Max. :0.61100 Max. :289.00 Max. :440.0 Max. :1.0390
pH sulphates alcohol quality color
Min. :2.720 Min. :0.2200 Min. : 8.00 Min. :3.000 Length:6497
1st Qu.:3.110 1st Qu.:0.4300 1st Qu.: 9.50 1st Qu.:5.000 Class :character
Median :3.210 Median :0.5100 Median :10.30 Median :6.000 Mode :character
Mean :3.219 Mean :0.5313 Mean :10.49 Mean :5.818
3rd Qu.:3.320 3rd Qu.:0.6000 3rd Qu.:11.30 3rd Qu.:6.000
Max. :4.010 Max. :2.0000 Max. :14.90 Max. :9.000
Let’s now start exploring the data one variable at a time. I’ve created a function below that will make a plot of two overlapping transparent histograms. It will be useful to compare the distributions for red wine and white wine.
plot_multi_histogram <- function(df, feature, label.column, alpha, binwidth) {
plt <- ggplot(df, aes(x=eval(parse(text=feature)), fill=eval(parse(text=label.column)))) +
geom_histogram(alpha=alpha, position="identity", color="black", binwidth=binwidth) +
#geom_density(alpha=alpha)# +
#geom_vline(aes(xintercept=mean(eval(parse(text=feature)))), color="black", linetype="dashed", size=1)# +
labs(x=feature, y = "Density")
plt + guides(fill=guide_legend(title=label.column))
}
# 1 - fixed acidity
summary(df$fixed.acidity)
Min. 1st Qu. Median Mean 3rd Qu. Max.
3.800 6.400 7.000 7.215 7.700 15.900
#table(df$fixed.acidity)
#sort(unique(df$fixed.acidity))
#ggplot(data = df, aes(x=fixed.acidity, fill=color)) + geom_histogram(alpha=0.2, position="identity")
plot_multi_histogram(df, "fixed.acidity", "color", alpha=0.4, binwidth=1)
# 2 - volatile acidity
summary(df$volatile.acidity)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.0800 0.2300 0.2900 0.3397 0.4000 1.5800
plot_multi_histogram(df, "volatile.acidity", "color", alpha=0.4, binwidth=0.1)
# 3 - citric acid
summary(df$citric.acid)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.0000 0.2500 0.3100 0.3186 0.3900 1.6600
plot_multi_histogram(df, "citric.acid", "color", alpha=0.4, binwidth=0.1)
# 4 - residual sugar
summary(df$residual.sugar)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.600 1.800 3.000 5.443 8.100 65.800
plot_multi_histogram(df, "residual.sugar", "color", alpha=0.4, binwidth=2)
# 5 - chlorides
summary(df$chlorides)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.00900 0.03800 0.04700 0.05603 0.06500 0.61100
plot_multi_histogram(df, "chlorides", "color", alpha=0.4, binwidth=0.01)
# 6 - free sulfur dioxide
summary(df$free.sulfur.dioxide)
Min. 1st Qu. Median Mean 3rd Qu. Max.
1.00 17.00 29.00 30.53 41.00 289.00
plot_multi_histogram(df, "free.sulfur.dioxide", "color", alpha=0.4, binwidth=10)
# 7 - total sulfur dioxide
summary(df$total.sulfur.dioxide)
Min. 1st Qu. Median Mean 3rd Qu. Max.
6.0 77.0 118.0 115.7 156.0 440.0
plot_multi_histogram(df, "total.sulfur.dioxide", "color", alpha=0.4, binwidth=10)
# 8 - density
summary(df$density)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.9871 0.9923 0.9949 0.9947 0.9970 1.0390
plot_multi_histogram(df, "density", "color", alpha=0.4, binwidth=0.001)
# 9 - pH
summary(df$pH)
Min. 1st Qu. Median Mean 3rd Qu. Max.
2.720 3.110 3.210 3.219 3.320 4.010
plot_multi_histogram(df, "pH", "color", alpha=0.4, binwidth=0.05)
# 10 - sulphates
summary(df$sulphates)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.2200 0.4300 0.5100 0.5313 0.6000 2.0000
plot_multi_histogram(df, "sulphates", "color", alpha=0.4, binwidth=0.1)
# 11 - alcohol
summary(df$alcohol)
Min. 1st Qu. Median Mean 3rd Qu. Max.
8.00 9.50 10.30 10.49 11.30 14.90
plot_multi_histogram(df, "alcohol", "color", alpha=0.4, binwidth=0.2)
# 12 - quality (score between 0 and 10)
summary(df$quality)
Min. 1st Qu. Median Mean 3rd Qu. Max.
3.000 5.000 6.000 5.818 6.000 9.000
plot_multi_histogram(df, "quality", "color", alpha=0.4, binwidth=1)
By looking at the data, we have learned the following:
There is more data on white wine than red wine, so we may need to rescale the data so that the model learns to classify red wine as well as white wine.
Several of the distributions are skewed, indicating the presence of long tails or outliers.
Now let’s look for correlations between features in the data with a ggpairs plot.
ggpairs(df, columns=1:12, mapping=ggplot2::aes(colour = color), lower=list(continuous='points')) + scale_color_manual(values=c("red", "yellow"))
The highest correlations we see are the following:
0.721 between free.sulfur.dioxide and total.sulfur.dioxide
0.687 between density and alcohol
0.553 between density and residual.sugar
0.495 between total.sulfur.dioxide and residual.sugar
0.459 between density and fixed.acidity
0.444 between alcohol and quality
0.414 between total.sulfur.dioxide and volatile.acidity
0.403 between free.sulfur.dioxide and residual.sugar
There is also a correlation between pH and volatile.acidity, but it’s different between red wine and white wine.
Cleary the alcohol content will be useful for our model because of its relatively high correlation with the wine quality rating (0.444).
We also see some outliers. Let’s take a closer look at some of the plots to figure out where these outliers are and if they ought to be removed. The most obvious outlier is the one in residual.sugar and density.
ggplot(df, aes(x=residual.sugar, y=density, color=as.factor(quality))) +
geom_point(size=6) + scale_color_brewer(palette = "Spectral")
The maximum has over twice as much sugar as the next largest point. It is also twice as far from the minimum density as the next most dense point. It is clearly an outlier and I believe it is safe to remove it from the data. We’ll do this by cutting out all points with residual.sugar > 40. The next largest point is also separated from most of the other points, but more so in density than in residual.sugar. The density is correlated with both alcohol (0.687) and residual.sugar (0.553). I think we can assume that much of the information about the density is included already in the alcohol and sugar variables, so we can remove density from the dataset.
Another outlier is in free.sulfur.dioxide and total.sulfur.dioxide.
ggplot(df, aes(x=free.sulfur.dioxide, y=total.sulfur.dioxide, color=as.factor(quality))) +
geom_point(size=6) + scale_color_brewer(palette = "Spectral")
This point has almost twice as much free sulfur dioxide as the next largest point. However, it is not much of an outlier on the other axis, total.sulfur.dioxide. It also has the minimum quality rating, which means it could have useful predictive power. There is a strong correlation between these two variables (0.721), which means it may be better to simply remove the variable free.sulfur.dioxide from the dataset rather than remove this data point.
Let’s apply the changes (remove sugar outlier & remove free.sulfur.dioxide (6) and density (8)).
df.cut <- df[df$residual.sugar < 40, c(1:5, 7, 9:13)]
Now we’ll redraw the plots.
ggpairs(df.cut, columns=1:10, mapping=ggplot2::aes(colour = color), lower=list(continuous='points')) + scale_color_manual(values=c("red", "yellow"))
There are still some points that tend to isolate themselves from most of the data, but not to such a degree as the outlier that we removed. They seem to simply follow the shape of a long-tailed distribution. This calls for a transformation of the data to make the distributions more normal.
Let’s begin the transformations by first calculating the skewness of the variables.
for (name in names(df.cut[1:9])) {
skew <- skewness(df.cut[name], na.rm = TRUE)
print(paste(name,skew))
}
[1] "fixed.acidity 1.72303425219973"
[1] "volatile.acidity 1.49292242490666"
[1] "citric.acid 0.47177011412896"
[1] "residual.sugar 1.1695985102309"
[1] "chlorides 5.39870943969405"
[1] "total.sulfur.dioxide -0.000889571569577466"
[1] "pH 0.387127126904609"
[1] "sulphates 1.79749476995869"
[1] "alcohol 0.565985435318878"
We now create a copy of the dataframe and apply transformations. The functions 1/x, sqrt, and log10 are useful for transforming positively skewed distributions into a more gaussian shape, depending on the severity of the skew. The following transformations give decent distributions.
df.transformed <- data.frame(df.cut)
df.transformed$fixed.acidity <- 1/(df.transformed$fixed.acidity)
df.transformed$sulphates <- 1/(df.transformed$sulphates)
df.transformed$volatile.acidity <- sqrt(1/(df.transformed$volatile.acidity))
df.transformed$residual.sugar <- log10(df.transformed$residual.sugar)
df.transformed$chlorides <- sqrt(1/(df.transformed$chlorides))
df.transformed$citric.acid <- sqrt(df.transformed$citric.acid + 0.1)
df.transformed$total.sulfur.dioxide <- sqrt(df.transformed$total.sulfur.dioxide)
Let’s plot a few histograms to see how normal the new distributions are.
plot_multi_histogram(df.transformed, "volatile.acidity", "color", alpha=0.4, binwidth=0.1)
plot_multi_histogram(df.transformed, "residual.sugar", "color", alpha=0.4, binwidth=0.1)
plot_multi_histogram(df.transformed, "chlorides", "color", alpha=0.4, binwidth=2)
plot_multi_histogram(df.transformed, "citric.acid", "color", alpha=0.4, binwidth=0.05)
We can now calculate the skewness again to see if it has improved.
for (name in names(df.transformed[1:9])) {
skew <- skewness(df.transformed[name], na.rm = TRUE)
print(paste(name,skew))
}
[1] "fixed.acidity -0.158566250269554"
[1] "volatile.acidity 0.248389283010552"
[1] "citric.acid -0.444243638771569"
[1] "residual.sugar 0.237148189575897"
[1] "chlorides 0.115471194185833"
[1] "total.sulfur.dioxide -0.652244809027631"
[1] "pH 0.387127126904609"
[1] "sulphates 0.441836626677462"
[1] "alcohol 0.565985435318878"
We can also plot a few scatter plots to look at two variables at once, to see how the distributions changed after the transformations and look for any more correlations. We can color-code the quality score.
ggplot(df.cut, aes(x=fixed.acidity, y=sulphates, color=as.factor(quality))) +
geom_point(size=6) + scale_color_brewer(palette = "Spectral")
ggplot(df.transformed, aes(x=fixed.acidity, y=sulphates, color=as.factor(quality))) +
geom_point(size=6) + scale_color_brewer(palette = "Spectral")
In this particular case it looks like the data have become more normally distributed. Let’s look at the full correlation plot again because it gives a good view of the entire dataset all at once.
ggpairs(df.transformed, columns=1:10, mapping=ggplot2::aes(colour = color), lower=list(continuous='points')) + scale_color_manual(values=c("red", "yellow"))
The data look very good now. However, there does not seem to be any easy way to predict the quality of the wine. None of the variables have a very high correlation with quality. The colored scatter plots that we have looked at do not have any obvious clustering of quality scores. Any significant correlations with quality must reside in combinations of more that two features, in a very nonlinear way. This makes the required application of a linear model unusual, but let’s see what we can do.
The data appear to be ready for the application of a simple test model to see if we can get the plsRglm package working correctly. Let’s transform the color variable into numeric factors and set up our X variables and Y target.
dataY <- factor(df.transformed$quality, ordered=TRUE)
dataX <- df.transformed[1:9]
df.transformed$color <- factor(df.transformed$color)
dataX['color'] <- as.numeric(df.transformed$color)
As stated previously, the data ought to be weighted by color so that the model isn’t biased towards white wine. We want it to learn to accurately predict the quality of either color. In this case, the difference in size of the two groups is not extreme, but it is still good practice to be aware of and take care of these issues.
weight.factor <- nrow(df.red)/nrow(df.white)
get_weight <- function(color.name) {
if (color.name == "white") {
return(weight.factor)
}
return(1.0)
}
prior.weights <- sapply(df.transformed$color, get_weight)
We now build a model using the required package plsRglm. See https://cran.r-project.org/web/packages/plsRglm/vignettes/plsRglm.pdf for more information on this package and specific applications. More information about the arguments can be found at https://www.rdocumentation.org/packages/plsRglm/versions/1.3.0/topics/plsRglm. We will train on 80% of the data and save 20% for testing later.
N.rows <- nrow(dataX)
train.fraction <- 0.8
N.train.rows <- floor(N.rows*train.fraction)
N.test.rows <- N.rows - N.train.rows
test.indices <- sample.int(N.rows, N.test.rows) # select N_test_rows randomly
dataX.test <- dataX[c(test.indices),]
dataX.train <- dataX[-c(test.indices),]
dataY.test <- dataY[c(test.indices)]
dataY.train <- dataY[-c(test.indices)]
prior.weights.test <- prior.weights[c(test.indices)]
prior.weights.train <- prior.weights[-c(test.indices)]
set.seed(123)
model.pls <- plsRglm(dataY.train,dataX.train,nt=10,limQ2set=.0975,
dataPredictY=dataX.train,modele="pls-glm-polr",family=NULL,typeVC="none",
EstimXNA=FALSE,scaleX=TRUE,scaleY=NULL,pvals.expli=FALSE,
alpha.pvals.expli=.05,MClassed=FALSE,tol_Xi=10^(-12),weights=prior.weights.train,
sparse=FALSE,sparseStop=TRUE,naive=FALSE,verbose=TRUE)
____************************************************____
Only naive DoF can be used with PLS GLM
Only naive DoF can be used with weighted PLS
Model: pls-glm-polr
Method: logistic
____Component____ 1 ____
____Component____ 2 ____
____Component____ 3 ____
____Component____ 4 ____
____Component____ 5 ____
No more significant predictors (<0.05) found
Warning only 5 components were thus extracted
____Predicting X without NA neither in X nor in Y____
****________________________________________________****
model.pls
Number of required components:
[1] 10
Number of successfully computed components:
[1] 5
Coefficients:
[,1]
3|4 7.43119294
4|5 9.54260785
5|6 12.61849044
6|7 15.21585817
7|8 17.57477370
8|9 21.60134406
fixed.acidity -1.97867791
volatile.acidity -1.67664252
citric.acid -0.45936171
residual.sugar -1.01052214
chlorides -0.24314098
total.sulfur.dioxide 0.02653666
pH -0.33875058
sulphates 0.50649502
alcohol -0.88540091
color 0.60749011
Information criteria and Fit statistics:
AIC BIC Missclassed Chi2_Pearson_Y
Nb_Comp_0 13287.31 13326.65 2928 6708.064
Nb_Comp_1 11936.06 11981.95 2579 5155.807
Nb_Comp_2 11634.73 11687.17 2437 4866.270
Nb_Comp_3 11486.87 11545.87 2419 4714.235
Nb_Comp_4 11435.80 11501.36 2377 4672.224
Nb_Comp_5 11431.05 11503.16 2370 4660.304
Let’s take a look at how well the model performs in predicting the quality of wines. We compare the predictions and observations with a table, a plot, and a calculation of the Mean Absolute Error (MAE).
predictions.train <- as.numeric(predict(model.pls$FinalModel))
observations.train <- as.numeric(dataY.train)
predictions.test <- as.numeric(predict(model.pls, newdata = dataX.test, type="class"))
observations.test <- as.numeric(dataY.test)
table(as.numeric(unlist(dataY.train)), predictions.train )
predictions.train
3 4 5 6
1 16 11 0 0
2 99 76 5 0
3 1008 676 15 1
4 477 1650 141 0
5 36 660 168 2
6 7 101 44 0
7 0 2 1 0
table(as.numeric(unlist(dataY.test)), predictions.test )
predictions.test
3 4 5
1 2 1 0
2 22 14 0
3 254 183 1
4 130 393 44
5 10 162 41
6 3 27 11
7 0 0 2
plot(observations.test, predictions.test)
mae(observations.train, predictions.train)
[1] 0.5132794
mae(observations.test, predictions.test)
[1] 0.52
This is not bad. It’s much better than randomly guessing numbers between 3 and 7, in which case I would expect an mae of about 1.8. But still, maybe we can do better.
A cross validation experiment demonstrates that 10 is an appropriate number of components for this model, given the training data. However, this takes a very long time to run. So I’m excluding it from this notebook.
#cv.modpls <- cv.plsRglm(dataY.train,dataX.train,nt=10,modele="pls-glm-polr",NK=5)
#cv.modpls
#res.cv.modpls=cvtable(summary(cv.modpls, MClassed = TRUE))
#plot(res.cv.modpls) # This suggests that 10 is the preferred number of components for the model.
Let’s try to improve the model by implementing an ensemble. Linear regression models are very robust and tend to get the same result every time, so making the same model again and again and averaging the results won’t do any good, unlike a random forest which can create thousands of different trees by simply changing the random see. Instead of bagging models together, let’s try a boosting technique. The first model will use the features to try and predict the quality score. The second model will use the features and the 1st model’s score to try and predict the true quality score, thus fitting to the difference, and so on.
dataX.train.2 <- data.frame(dataX.train)
dataX.train.2["prediction1"] <- as.numeric( predict(model.pls$FinalModel) )
dataX.test.2 <- data.frame(dataX.test)
dataX.test.2["prediction1"] <- as.numeric( predict(model.pls, newdata = dataX.test, type="class") )
model.pls.2 <- plsRglm(dataY.train,dataX.train.2,nt=10,limQ2set=.0975,
dataPredictY=dataX.train.2,modele="pls-glm-polr",family=NULL,typeVC="none",
EstimXNA=FALSE,scaleX=TRUE,scaleY=NULL,pvals.expli=FALSE,
alpha.pvals.expli=.05,MClassed=FALSE,tol_Xi=10^(-12),weights=prior.weights.train)
____************************************************____
Only naive DoF can be used with weighted PLS
Model: pls-glm-polr
Method: logistic
____Component____ 1 ____
____Component____ 2 ____
____Component____ 3 ____
____Component____ 4 ____
____Component____ 5 ____
____Component____ 6 ____
____Component____ 7 ____
____Component____ 8 ____
____Component____ 9 ____
____Component____ 10 ____
____Predicting X without NA neither in X nor in Y____
****________________________________________________****
model.pls.2
Number of required components:
[1] 10
Number of successfully computed components:
[1] 10
Coefficients:
[,1]
3|4 6.87068810
4|5 8.98709904
5|6 12.06270840
6|7 14.66104119
7|8 17.02131062
8|9 21.04616144
fixed.acidity -1.75316521
volatile.acidity -1.66815402
citric.acid -0.33562439
residual.sugar -0.98455757
chlorides -0.22967034
total.sulfur.dioxide 0.01493810
pH -0.15275462
sulphates 0.49511138
alcohol -0.89712185
color 0.65638385
prediction1 -0.01754957
Information criteria and Fit statistics:
AIC BIC Missclassed Chi2_Pearson_Y
Nb_Comp_0 13287.31 13326.65 2928 6708.064
Nb_Comp_1 11737.32 11783.21 2391 4675.422
Nb_Comp_2 11606.70 11659.15 2371 4383.452
Nb_Comp_3 11496.50 11555.50 2378 4415.080
Nb_Comp_4 11431.92 11497.48 2377 4681.173
Nb_Comp_5 11429.96 11502.07 2371 4646.434
Nb_Comp_6 11431.47 11510.14 2368 4653.847
Nb_Comp_7 11433.39 11518.62 2372 4647.880
Nb_Comp_8 11435.39 11527.17 2374 4649.227
Nb_Comp_9 11437.38 11535.72 2372 4649.341
Nb_Comp_10 11439.38 11544.27 2371 4649.378
We now calculate the MAE to see how well the second model performed.
predictions.train <- as.numeric(predict(model.pls.2$FinalModel))
observations.train <- as.numeric(dataY.train)
predictions.test <- as.numeric(predict(model.pls.2, newdata = dataX.test.2, type="class"))
observations.test <- as.numeric(dataY.test)
mae(observations.train, predictions.train)
[1] 0.5134719
mae(observations.test, predictions.test)
[1] 0.5161538
Boosting in this manner offered no significant improvement in the MAE.
We could try a boosted ensemble where we just do one variable at a time and then fit the prediction error with each successive model, but I think that would be just like multivariate regression, but not as good.
Let’s continue trying a boosted ensemble with all of the variables, including the result of the previous model to make the next predictions. But instead of trying to fit the quality values at each step, we can try to fit the difference between the quality values and the previous prediction. Let’s try it.
number.of.models <- 3
models.pls <- vector(mode = "list", length = number.of.models) # empty list of models
errors <- vector(mode = "list", length = number.of.models) # empty list of errors
predictions <- as.numeric(dataY.train)*0.0
targets <- as.numeric(dataY.train)
for (i in 1:number.of.models) {
dataX.this <- data.frame(dataX.train)
if (i > 1) {
dataX.this["prediction1"] <- predictions
}
targets <- targets - predictions
model.this <- plsRglm(targets,dataX.this,nt=10,limQ2set=.0975,
dataPredictY=dataX.this,modele="pls-glm-polr",family=NULL,typeVC="none",
EstimXNA=FALSE,scaleX=TRUE,scaleY=NULL,pvals.expli=FALSE,
alpha.pvals.expli=.05,MClassed=FALSE,tol_Xi=10^(-12),weights=prior.weights.train,
sparse=FALSE,sparseStop=TRUE,naive=FALSE,verbose=TRUE)
#model_this
predictions <- as.numeric(predict(model.this$FinalModel))
observations <- as.numeric(targets)
errors[i] <- mae(observations, predictions)
models_pls[i] <- model_this
print(errors[i])
}
____************************************************____
Only naive DoF can be used with PLS GLM
Only naive DoF can be used with weighted PLS
Model: pls-glm-polr
Method: logistic
____Component____ 1 ____
____Component____ 2 ____
____Component____ 3 ____
____Component____ 4 ____
____Component____ 5 ____
No more significant predictors (<0.05) found
Warning only 5 components were thus extracted
____Predicting X without NA neither in X nor in Y____
****________________________________________________****
number of items to replace is not a multiple of replacement length
[[1]]
[1] 0.5132794
____************************************************____
Only naive DoF can be used with PLS GLM
Only naive DoF can be used with weighted PLS
Model: pls-glm-polr
Method: logistic
____Component____ 1 ____
____Component____ 2 ____
____Component____ 3 ____
____Component____ 4 ____
____Component____ 5 ____
____Component____ 6 ____
No more significant predictors (<0.05) found
Warning only 6 components were thus extracted
____Predicting X without NA neither in X nor in Y____
****________________________________________________****
number of items to replace is not a multiple of replacement length
[[1]]
[1] 3.961124
____************************************************____
Only naive DoF can be used with PLS GLM
Only naive DoF can be used with weighted PLS
Model: pls-glm-polr
Method: logistic
____Component____ 1 ____
____Component____ 2 ____
No more significant predictors (<0.05) found
Warning only 2 components were thus extracted
____Predicting X without NA neither in X nor in Y____
****________________________________________________****
number of items to replace is not a multiple of replacement length
[[1]]
[1] 7.946112
This is resulting in models that get progressively worse. I see now that I was on the wrong track. The data are too noisy, so by boosting a noisy model I just get an even noisier model. What I can do is bag the models instead, which is what I didn’t want to do at first because every model would be the same, but actually that’s only true if I use the same data each time. I could split up the dataset into N chunks and fit each one separately, and then I’ll use the results all together for the final prediction. But that’s basically just multivariate linear regression on the full dataset, only worse.
Let’s try another approach. Since the plsRglm model is designed to handle high dimensional data with few rows, I’m going to increase the dimensionality of the dataset. This will allow previously nonlinear quadratic properties to be described linearly by the plsRglm model.
If I think that the relationship between the data and quality is nonlinear then I just need to create nonlinear features. If I want xy in the model instead of just ax + by I can make a column xy by multiplying two columns together. I’ll multiply every continuous feature by every other continuous feature to get new quadratic columns.
dataX.nonlinear <- data.frame(dataX)
for (feature1 in names(dataX.nonlinear[1:9])) {
for (feature2 in names(dataX.nonlinear[1:9])) {
new.feature <- paste(substring(feature1,1,2), substring(feature2,1,2), sep=".")
dataX.nonlinear[new.feature] <- dataX.nonlinear[feature1]*dataX.nonlinear[feature2]
}
}
names(dataX_nonlinear)
[1] "fixed.acidity" "volatile.acidity" "citric.acid"
[4] "residual.sugar" "chlorides" "total.sulfur.dioxide"
[7] "pH" "sulphates" "alcohol"
[10] "color" "fi.fi" "fi.vo"
[13] "fi.ci" "fi.re" "fi.ch"
[16] "fi.to" "fi.pH" "fi.su"
[19] "fi.al" "vo.fi" "vo.vo"
[22] "vo.ci" "vo.re" "vo.ch"
[25] "vo.to" "vo.pH" "vo.su"
[28] "vo.al" "ci.fi" "ci.vo"
[31] "ci.ci" "ci.re" "ci.ch"
[34] "ci.to" "ci.pH" "ci.su"
[37] "ci.al" "re.fi" "re.vo"
[40] "re.ci" "re.re" "re.ch"
[43] "re.to" "re.pH" "re.su"
[46] "re.al" "ch.fi" "ch.vo"
[49] "ch.ci" "ch.re" "ch.ch"
[52] "ch.to" "ch.pH" "ch.su"
[55] "ch.al" "to.fi" "to.vo"
[58] "to.ci" "to.re" "to.ch"
[61] "to.to" "to.pH" "to.su"
[64] "to.al" "pH.fi" "pH.vo"
[67] "pH.ci" "pH.re" "pH.ch"
[70] "pH.to" "pH.pH" "pH.su"
[73] "pH.al" "su.fi" "su.vo"
[76] "su.ci" "su.re" "su.ch"
[79] "su.to" "su.pH" "su.su"
[82] "su.al" "al.fi" "al.vo"
[85] "al.ci" "al.re" "al.ch"
[88] "al.to" "al.pH" "al.su"
[91] "al.al" "r.fixed.acidity" "w.fixed.acidity"
[94] "r.volatile.acidity" "w.volatile.acidity" "r.citric.acid"
[97] "w.citric.acid" "r.residual.sugar" "w.residual.sugar"
[100] "r.chlorides" "w.chlorides" "r.total.sulfur.dioxide"
[103] "w.total.sulfur.dioxide" "r.pH" "w.pH"
[106] "r.sulphates" "w.sulphates" "r.alcohol"
[109] "w.alcohol" "r.fi.fi" "w.fi.fi"
[112] "r.fi.vo" "w.fi.vo" "r.fi.ci"
[115] "w.fi.ci" "r.fi.re" "w.fi.re"
[118] "r.fi.ch" "w.fi.ch" "r.fi.to"
[121] "w.fi.to" "r.fi.pH" "w.fi.pH"
[124] "r.fi.su" "w.fi.su" "r.fi.al"
[127] "w.fi.al" "r.vo.fi" "w.vo.fi"
[130] "r.vo.vo" "w.vo.vo" "r.vo.ci"
[133] "w.vo.ci" "r.vo.re" "w.vo.re"
[136] "r.vo.ch" "w.vo.ch" "r.vo.to"
[139] "w.vo.to" "r.vo.pH" "w.vo.pH"
[142] "r.vo.su" "w.vo.su" "r.vo.al"
[145] "w.vo.al" "r.ci.fi" "w.ci.fi"
[148] "r.ci.vo" "w.ci.vo" "r.ci.ci"
[151] "w.ci.ci" "r.ci.re" "w.ci.re"
[154] "r.ci.ch" "w.ci.ch" "r.ci.to"
[157] "w.ci.to" "r.ci.pH" "w.ci.pH"
[160] "r.ci.su" "w.ci.su" "r.ci.al"
[163] "w.ci.al" "r.re.fi" "w.re.fi"
[166] "r.re.vo" "w.re.vo" "r.re.ci"
[169] "w.re.ci" "r.re.re" "w.re.re"
[172] "r.re.ch" "w.re.ch" "r.re.to"
[175] "w.re.to" "r.re.pH" "w.re.pH"
[178] "r.re.su" "w.re.su" "r.re.al"
[181] "w.re.al" "r.ch.fi" "w.ch.fi"
[184] "r.ch.vo" "w.ch.vo" "r.ch.ci"
[187] "w.ch.ci" "r.ch.re" "w.ch.re"
[190] "r.ch.ch" "w.ch.ch" "r.ch.to"
[193] "w.ch.to" "r.ch.pH" "w.ch.pH"
[196] "r.ch.su" "w.ch.su" "r.ch.al"
[199] "w.ch.al" "r.to.fi" "w.to.fi"
[202] "r.to.vo" "w.to.vo" "r.to.ci"
[205] "w.to.ci" "r.to.re" "w.to.re"
[208] "r.to.ch" "w.to.ch" "r.to.to"
[211] "w.to.to" "r.to.pH" "w.to.pH"
[214] "r.to.su" "w.to.su" "r.to.al"
[217] "w.to.al" "r.pH.fi" "w.pH.fi"
[220] "r.pH.vo" "w.pH.vo" "r.pH.ci"
[223] "w.pH.ci" "r.pH.re" "w.pH.re"
[226] "r.pH.ch" "w.pH.ch" "r.pH.to"
[229] "w.pH.to" "r.pH.pH" "w.pH.pH"
[232] "r.pH.su" "w.pH.su" "r.pH.al"
[235] "w.pH.al" "r.su.fi" "w.su.fi"
[238] "r.su.vo" "w.su.vo" "r.su.ci"
[241] "w.su.ci" "r.su.re" "w.su.re"
[244] "r.su.ch" "w.su.ch" "r.su.to"
[247] "w.su.to" "r.su.pH" "w.su.pH"
[250] "r.su.su" "w.su.su" "r.su.al"
[253] "w.su.al" "r.al.fi" "w.al.fi"
[256] "r.al.vo" "w.al.vo" "r.al.ci"
[259] "w.al.ci" "r.al.re" "w.al.re"
[262] "r.al.ch" "w.al.ch" "r.al.to"
[265] "w.al.to" "r.al.pH" "w.al.pH"
[268] "r.al.su" "w.al.su" "r.al.al"
[271] "w.al.al"
I should also separate the red and white wine features, because they appear to be significantly different.
for (feature in names(dataX.nonlinear[c(1:9, 11:91)])) {
r.feature <- paste("r", feature, sep=".")
w.feature <- paste("w", feature, sep=".")
dataX.nonlinear[r.feature] <- with(dataX.nonlinear,
ifelse(color == 1,
eval(parse(text=feature)),
0.0)
)
dataX.nonlinear[w.feature] <- with(dataX.nonlinear,
ifelse(color == 2,
eval(parse(text=feature)),
0.0)
)
}
Let’s look at the new dataframe features.
names(dataX_nonlinear)
[1] "fixed.acidity" "volatile.acidity" "citric.acid"
[4] "residual.sugar" "chlorides" "total.sulfur.dioxide"
[7] "pH" "sulphates" "alcohol"
[10] "color" "fi.fi" "fi.vo"
[13] "fi.ci" "fi.re" "fi.ch"
[16] "fi.to" "fi.pH" "fi.su"
[19] "fi.al" "vo.fi" "vo.vo"
[22] "vo.ci" "vo.re" "vo.ch"
[25] "vo.to" "vo.pH" "vo.su"
[28] "vo.al" "ci.fi" "ci.vo"
[31] "ci.ci" "ci.re" "ci.ch"
[34] "ci.to" "ci.pH" "ci.su"
[37] "ci.al" "re.fi" "re.vo"
[40] "re.ci" "re.re" "re.ch"
[43] "re.to" "re.pH" "re.su"
[46] "re.al" "ch.fi" "ch.vo"
[49] "ch.ci" "ch.re" "ch.ch"
[52] "ch.to" "ch.pH" "ch.su"
[55] "ch.al" "to.fi" "to.vo"
[58] "to.ci" "to.re" "to.ch"
[61] "to.to" "to.pH" "to.su"
[64] "to.al" "pH.fi" "pH.vo"
[67] "pH.ci" "pH.re" "pH.ch"
[70] "pH.to" "pH.pH" "pH.su"
[73] "pH.al" "su.fi" "su.vo"
[76] "su.ci" "su.re" "su.ch"
[79] "su.to" "su.pH" "su.su"
[82] "su.al" "al.fi" "al.vo"
[85] "al.ci" "al.re" "al.ch"
[88] "al.to" "al.pH" "al.su"
[91] "al.al" "r.fixed.acidity" "w.fixed.acidity"
[94] "r.volatile.acidity" "w.volatile.acidity" "r.citric.acid"
[97] "w.citric.acid" "r.residual.sugar" "w.residual.sugar"
[100] "r.chlorides" "w.chlorides" "r.total.sulfur.dioxide"
[103] "w.total.sulfur.dioxide" "r.pH" "w.pH"
[106] "r.sulphates" "w.sulphates" "r.alcohol"
[109] "w.alcohol" "r.fi.fi" "w.fi.fi"
[112] "r.fi.vo" "w.fi.vo" "r.fi.ci"
[115] "w.fi.ci" "r.fi.re" "w.fi.re"
[118] "r.fi.ch" "w.fi.ch" "r.fi.to"
[121] "w.fi.to" "r.fi.pH" "w.fi.pH"
[124] "r.fi.su" "w.fi.su" "r.fi.al"
[127] "w.fi.al" "r.vo.fi" "w.vo.fi"
[130] "r.vo.vo" "w.vo.vo" "r.vo.ci"
[133] "w.vo.ci" "r.vo.re" "w.vo.re"
[136] "r.vo.ch" "w.vo.ch" "r.vo.to"
[139] "w.vo.to" "r.vo.pH" "w.vo.pH"
[142] "r.vo.su" "w.vo.su" "r.vo.al"
[145] "w.vo.al" "r.ci.fi" "w.ci.fi"
[148] "r.ci.vo" "w.ci.vo" "r.ci.ci"
[151] "w.ci.ci" "r.ci.re" "w.ci.re"
[154] "r.ci.ch" "w.ci.ch" "r.ci.to"
[157] "w.ci.to" "r.ci.pH" "w.ci.pH"
[160] "r.ci.su" "w.ci.su" "r.ci.al"
[163] "w.ci.al" "r.re.fi" "w.re.fi"
[166] "r.re.vo" "w.re.vo" "r.re.ci"
[169] "w.re.ci" "r.re.re" "w.re.re"
[172] "r.re.ch" "w.re.ch" "r.re.to"
[175] "w.re.to" "r.re.pH" "w.re.pH"
[178] "r.re.su" "w.re.su" "r.re.al"
[181] "w.re.al" "r.ch.fi" "w.ch.fi"
[184] "r.ch.vo" "w.ch.vo" "r.ch.ci"
[187] "w.ch.ci" "r.ch.re" "w.ch.re"
[190] "r.ch.ch" "w.ch.ch" "r.ch.to"
[193] "w.ch.to" "r.ch.pH" "w.ch.pH"
[196] "r.ch.su" "w.ch.su" "r.ch.al"
[199] "w.ch.al" "r.to.fi" "w.to.fi"
[202] "r.to.vo" "w.to.vo" "r.to.ci"
[205] "w.to.ci" "r.to.re" "w.to.re"
[208] "r.to.ch" "w.to.ch" "r.to.to"
[211] "w.to.to" "r.to.pH" "w.to.pH"
[214] "r.to.su" "w.to.su" "r.to.al"
[217] "w.to.al" "r.pH.fi" "w.pH.fi"
[220] "r.pH.vo" "w.pH.vo" "r.pH.ci"
[223] "w.pH.ci" "r.pH.re" "w.pH.re"
[226] "r.pH.ch" "w.pH.ch" "r.pH.to"
[229] "w.pH.to" "r.pH.pH" "w.pH.pH"
[232] "r.pH.su" "w.pH.su" "r.pH.al"
[235] "w.pH.al" "r.su.fi" "w.su.fi"
[238] "r.su.vo" "w.su.vo" "r.su.ci"
[241] "w.su.ci" "r.su.re" "w.su.re"
[244] "r.su.ch" "w.su.ch" "r.su.to"
[247] "w.su.to" "r.su.pH" "w.su.pH"
[250] "r.su.su" "w.su.su" "r.su.al"
[253] "w.su.al" "r.al.fi" "w.al.fi"
[256] "r.al.vo" "w.al.vo" "r.al.ci"
[259] "w.al.ci" "r.al.re" "w.al.re"
[262] "r.al.ch" "w.al.ch" "r.al.to"
[265] "w.al.to" "r.al.pH" "w.al.pH"
[268] "r.al.su" "w.al.su" "r.al.al"
[271] "w.al.al"
Now let’s remove all the columns that don’t start with r. or w.
dataX.nonlinear.final <- dataX.nonlinear[c(92:271)]
names(dataX.nonlinear.final)
[1] "r.fixed.acidity" "w.fixed.acidity" "r.volatile.acidity"
[4] "w.volatile.acidity" "r.citric.acid" "w.citric.acid"
[7] "r.residual.sugar" "w.residual.sugar" "r.chlorides"
[10] "w.chlorides" "r.total.sulfur.dioxide" "w.total.sulfur.dioxide"
[13] "r.pH" "w.pH" "r.sulphates"
[16] "w.sulphates" "r.alcohol" "w.alcohol"
[19] "r.fi.fi" "w.fi.fi" "r.fi.vo"
[22] "w.fi.vo" "r.fi.ci" "w.fi.ci"
[25] "r.fi.re" "w.fi.re" "r.fi.ch"
[28] "w.fi.ch" "r.fi.to" "w.fi.to"
[31] "r.fi.pH" "w.fi.pH" "r.fi.su"
[34] "w.fi.su" "r.fi.al" "w.fi.al"
[37] "r.vo.fi" "w.vo.fi" "r.vo.vo"
[40] "w.vo.vo" "r.vo.ci" "w.vo.ci"
[43] "r.vo.re" "w.vo.re" "r.vo.ch"
[46] "w.vo.ch" "r.vo.to" "w.vo.to"
[49] "r.vo.pH" "w.vo.pH" "r.vo.su"
[52] "w.vo.su" "r.vo.al" "w.vo.al"
[55] "r.ci.fi" "w.ci.fi" "r.ci.vo"
[58] "w.ci.vo" "r.ci.ci" "w.ci.ci"
[61] "r.ci.re" "w.ci.re" "r.ci.ch"
[64] "w.ci.ch" "r.ci.to" "w.ci.to"
[67] "r.ci.pH" "w.ci.pH" "r.ci.su"
[70] "w.ci.su" "r.ci.al" "w.ci.al"
[73] "r.re.fi" "w.re.fi" "r.re.vo"
[76] "w.re.vo" "r.re.ci" "w.re.ci"
[79] "r.re.re" "w.re.re" "r.re.ch"
[82] "w.re.ch" "r.re.to" "w.re.to"
[85] "r.re.pH" "w.re.pH" "r.re.su"
[88] "w.re.su" "r.re.al" "w.re.al"
[91] "r.ch.fi" "w.ch.fi" "r.ch.vo"
[94] "w.ch.vo" "r.ch.ci" "w.ch.ci"
[97] "r.ch.re" "w.ch.re" "r.ch.ch"
[100] "w.ch.ch" "r.ch.to" "w.ch.to"
[103] "r.ch.pH" "w.ch.pH" "r.ch.su"
[106] "w.ch.su" "r.ch.al" "w.ch.al"
[109] "r.to.fi" "w.to.fi" "r.to.vo"
[112] "w.to.vo" "r.to.ci" "w.to.ci"
[115] "r.to.re" "w.to.re" "r.to.ch"
[118] "w.to.ch" "r.to.to" "w.to.to"
[121] "r.to.pH" "w.to.pH" "r.to.su"
[124] "w.to.su" "r.to.al" "w.to.al"
[127] "r.pH.fi" "w.pH.fi" "r.pH.vo"
[130] "w.pH.vo" "r.pH.ci" "w.pH.ci"
[133] "r.pH.re" "w.pH.re" "r.pH.ch"
[136] "w.pH.ch" "r.pH.to" "w.pH.to"
[139] "r.pH.pH" "w.pH.pH" "r.pH.su"
[142] "w.pH.su" "r.pH.al" "w.pH.al"
[145] "r.su.fi" "w.su.fi" "r.su.vo"
[148] "w.su.vo" "r.su.ci" "w.su.ci"
[151] "r.su.re" "w.su.re" "r.su.ch"
[154] "w.su.ch" "r.su.to" "w.su.to"
[157] "r.su.pH" "w.su.pH" "r.su.su"
[160] "w.su.su" "r.su.al" "w.su.al"
[163] "r.al.fi" "w.al.fi" "r.al.vo"
[166] "w.al.vo" "r.al.ci" "w.al.ci"
[169] "r.al.re" "w.al.re" "r.al.ch"
[172] "w.al.ch" "r.al.to" "w.al.to"
[175] "r.al.pH" "w.al.pH" "r.al.su"
[178] "w.al.su" "r.al.al" "w.al.al"
Now we’re ready to fit the data with a model. We no longer need the prior weights because the red and white wines now live in different dimensions of the data.
dataX.nonlinear.test <- dataX.nonlinear.final[c(test.indices),]
dataX.nonlinear.train <- dataX.nonlinear.final[-c(test.indices),]
set.seed(123)
model.nonlinear <- plsRglm(dataY.train,dataX.nonlinear.train,nt=10,limQ2set=.0975,
dataPredictY=dataX.nonlinear.train,modele="pls-glm-polr",family=NULL,typeVC="none",
EstimXNA=FALSE,scaleX=TRUE,scaleY=NULL,pvals.expli=FALSE,
alpha.pvals.expli=.05,MClassed=FALSE,tol_Xi=10^(-12),
sparse=FALSE,sparseStop=TRUE,naive=FALSE,verbose=TRUE)
____************************************************____
Only naive DoF can be used with PLS GLM
Model: pls-glm-polr
Method: logistic
____Component____ 1 ____
____Component____ 2 ____
____Component____ 3 ____
____Component____ 4 ____
____Component____ 5 ____
____Component____ 6 ____
____Component____ 7 ____
____Component____ 8 ____
____Component____ 9 ____
____Component____ 10 ____
____Predicting X without NA neither in X nor in Y____
****________________________________________________****
model.nonlinear
Number of required components:
[1] 10
Number of successfully computed components:
[1] 10
Coefficients:
[,1]
3|4 0.6581066007
4|5 2.8069273439
5|6 5.9639681915
6|7 8.6308236355
7|8 11.0496809584
8|9 15.1074411771
r.fixed.acidity -6.4991486621
w.fixed.acidity 3.1757588445
r.volatile.acidity -1.1204206789
w.volatile.acidity -0.1173085603
r.citric.acid -0.4850643745
w.citric.acid 0.5493441696
r.residual.sugar -0.4098775709
w.residual.sugar -0.0946582713
r.chlorides -0.3238003724
w.chlorides 0.1284522445
r.total.sulfur.dioxide -0.0195994713
w.total.sulfur.dioxide 0.0464197462
r.pH -0.5566304720
w.pH 0.5589080659
r.sulphates 0.0028262465
w.sulphates 0.3083566706
r.alcohol -0.3079347055
w.alcohol 0.0040333633
r.fi.fi 19.0799599790
w.fi.fi 3.8598794104
r.fi.vo 0.3842029687
w.fi.vo -0.5126331783
r.fi.ci -0.5421993053
w.fi.ci -6.1741461029
r.fi.re 0.3218962198
w.fi.re 2.6585273010
r.fi.ch 0.4782511508
w.fi.ch -0.1939124458
r.fi.to 0.0710756054
w.fi.to 0.0988255689
r.fi.pH 0.5802452150
w.fi.pH 0.1209011834
r.fi.su 0.4948137929
w.fi.su 0.5092107439
r.fi.al -0.0507531332
w.fi.al -0.5545008159
r.vo.fi 0.3842029687
w.vo.fi -0.5126331783
r.vo.vo 0.1170197299
w.vo.vo -0.0385453479
r.vo.ci 0.3438101350
w.vo.ci -0.1219763557
r.vo.re -0.0027208287
w.vo.re -0.4025007914
r.vo.ch -0.0433883320
w.vo.ch 0.0384529478
r.vo.to 0.0268454217
w.vo.to -0.0267501369
r.vo.pH 0.0049481418
w.vo.pH -0.0531237124
r.vo.su -0.0961405848
w.vo.su -0.0039936237
r.vo.al -0.0468435522
w.vo.al -0.0005152789
r.ci.fi -0.5421993053
w.ci.fi -6.1741461029
r.ci.vo 0.3438101350
w.ci.vo -0.1219763557
r.ci.ci 0.5057359268
w.ci.ci 0.7795513101
r.ci.re 0.4001478801
w.ci.re 0.6037981814
r.ci.ch -0.1493637136
w.ci.ch 0.0535397468
r.ci.to 0.0980175319
w.ci.to 0.0137857944
r.ci.pH 0.2304521797
w.ci.pH -0.1051006471
r.ci.su -0.1180163286
w.ci.su -0.0802514327
r.ci.al -0.0661453037
w.ci.al -0.0589079435
r.re.fi 0.3218962198
w.re.fi 2.6585273010
r.re.vo -0.0027208287
w.re.vo -0.4025007914
r.re.ci 0.4001478801
w.re.ci 0.6037981814
r.re.re 0.2272403007
w.re.re 0.1665229157
r.re.ch 0.0394333828
w.re.ch -0.1074187916
r.re.to 0.0339822809
w.re.to 0.0259064587
r.re.pH 0.0179904207
w.re.pH 0.1197861532
r.re.su -0.2167001237
w.re.su -0.4459108286
r.re.al -0.0230056805
w.re.al 0.0282968045
r.ch.fi 0.4782511508
w.ch.fi -0.1939124458
r.ch.vo -0.0433883320
w.ch.vo 0.0384529478
r.ch.ci -0.1493637136
w.ch.ci 0.0535397468
r.ch.re 0.0394333828
w.ch.re -0.1074187916
r.ch.ch -0.0023186076
w.ch.ch 0.0032009357
r.ch.to -0.0053942521
w.ch.to -0.0072973610
r.ch.pH 0.0288025786
w.ch.pH -0.0273572260
r.ch.su 0.0407031161
w.ch.su 0.0394381122
r.ch.al -0.0070940742
w.ch.al -0.0125756885
r.to.fi 0.0710756054
w.to.fi 0.0988255689
r.to.vo 0.0268454217
w.to.vo -0.0267501369
r.to.ci 0.0980175319
w.to.ci 0.0137857944
r.to.re 0.0339822809
w.to.re 0.0259064587
r.to.ch -0.0053942521
w.to.ch -0.0072973610
r.to.to 0.0033288356
w.to.to 0.0057165686
r.to.pH 0.0036133358
w.to.pH 0.0064476579
r.to.su -0.0440206016
w.to.su -0.0131449260
r.to.al -0.0012642612
w.to.al -0.0054600617
r.pH.fi 0.5802452150
w.pH.fi 0.1209011834
r.pH.vo 0.0049481418
w.pH.vo -0.0531237124
r.pH.ci 0.2304521797
w.pH.ci -0.1051006471
r.pH.re 0.0179904207
w.pH.re 0.1197861532
r.pH.ch 0.0288025786
w.pH.ch -0.0273572260
r.pH.to 0.0036133358
w.pH.to 0.0064476579
r.pH.pH 0.0704027598
w.pH.pH 0.0021288275
r.pH.su 0.1881574061
w.pH.su 0.0674640554
r.pH.al -0.0128353131
w.pH.al -0.0410647627
r.su.fi 0.4948137929
w.su.fi 0.5092107439
r.su.vo -0.0961405848
w.su.vo -0.0039936237
r.su.ci -0.1180163286
w.su.ci -0.0802514327
r.su.re -0.2167001237
w.su.re -0.4459108286
r.su.ch 0.0407031161
w.su.ch 0.0394381122
r.su.to -0.0440206016
w.su.to -0.0131449260
r.su.pH 0.1881574061
w.su.pH 0.0674640554
r.su.su 0.2569719690
w.su.su 0.0112073044
r.su.al 0.0192420661
w.su.al -0.0009256702
r.al.fi -0.0507531332
w.al.fi -0.5545008159
r.al.vo -0.0468435522
w.al.vo -0.0005152789
r.al.ci -0.0661453037
w.al.ci -0.0589079435
r.al.re -0.0230056805
w.al.re 0.0282968045
r.al.ch -0.0070940742
w.al.ch -0.0125756885
r.al.to -0.0012642612
w.al.to -0.0054600617
r.al.pH -0.0128353131
w.al.pH -0.0410647627
r.al.su 0.0192420661
w.al.su -0.0009256702
r.al.al -0.0066022795
w.al.al -0.0096352398
Information criteria and Fit statistics:
AIC BIC Missclassed Chi2_Pearson_Y
Nb_Comp_0 13287.31 13326.65 2928 6708.064
Nb_Comp_1 13162.59 13208.48 2908 6463.251
Nb_Comp_2 11844.55 11897.00 2526 5082.886
Nb_Comp_3 11514.18 11573.18 2423 4732.986
Nb_Comp_4 11403.17 11468.72 2372 4649.119
Nb_Comp_5 11354.12 11426.23 2348 4640.854
Nb_Comp_6 11344.02 11422.69 2333 4629.086
Nb_Comp_7 11333.53 11418.75 2359 4616.520
Nb_Comp_8 11301.54 11393.32 2357 4582.407
Nb_Comp_9 11247.62 11345.95 2380 4549.792
Nb_Comp_10 11221.78 11326.67 2379 4533.511
Now let’s see how well it did.
predictions.train <- as.numeric(predict(model.nonlinear$FinalModel))
observations.train <- as.numeric(dataY.train)
predictions.test <- as.numeric(predict(model.nonlinear, newdata = dataX.nonlinear.test, type="class"))
observations.test <- as.numeric(dataY.test)
mae(observations.train, predictions.train)
[1] 0.5086605
mae(observations.test, predictions.test)
[1] 0.4953846
The MAE has improved slightly, by about 1%. Let’s try this again, but restricted to the variables which showed the highest correlation with quality.
for (feature in names(dataX.nonlinear.train)) {
feature.cor <- cor(dataX.nonlinear.train[feature], as.numeric(dataY.train))
if (abs(feature.cor) > 0.2) {
print(feature)
print(feature.cor)
}
}
[1] "w.fi.ch"
[,1]
w.fi.ch 0.201826
[1] "w.fi.al"
[,1]
w.fi.al 0.2183314
[1] "w.vo.ch"
[,1]
w.vo.ch 0.2208955
[1] "w.vo.al"
[,1]
w.vo.al 0.2374277
[1] "w.ch.fi"
[,1]
w.ch.fi 0.201826
[1] "w.ch.vo"
[,1]
w.ch.vo 0.2208955
[1] "w.ch.ch"
[,1]
w.ch.ch 0.2257812
[1] "w.ch.al"
[,1]
w.ch.al 0.2562421
[1] "w.pH.al"
[,1]
w.pH.al 0.2063902
[1] "w.al.fi"
[,1]
w.al.fi 0.2183314
[1] "w.al.vo"
[,1]
w.al.vo 0.2374277
[1] "w.al.ch"
[,1]
w.al.ch 0.2562421
[1] "w.al.pH"
[,1]
w.al.pH 0.2063902
[1] "w.al.al"
[,1]
w.al.al 0.2699345
These are the best correlations we see. They are very low and I don’t believe they will be useful. Plus, they are all white wine features.
Let’s try another approach. We can bin the features, and let every bin become its own feature. This will create a new dimension, and thus a different fitted slope, for every bin, thus allowing nonlinear features to be described by the linear model.
We now transform the data points row by row into a more sparse dataset of binned features.
dataX.expanded <- data.frame(dataX)
There were 24 warnings (use warnings() to see them)
nbins.default <- 20
nbins <- nbins.default
for (feature in names(dataX.expanded)) {
if (feature == "color") {
nbins <- 2
} else {
nbins <- nbins.default
}
# map value of feature to bin number
xmin <- 0.99*min(dataX.expanded[feature])
xmax <- 1.01*max(dataX.expanded[feature])
slope <- nbins/(xmax - xmin)
feature.bin <- paste(feature, "bin", sep="")
dataX.expanded[feature.bin] <- with(dataX.expanded, floor( slope*( eval(parse(text=feature)) - xmin) ) )
# create new feature for each bin
for (i in 1:nbins) {
new.feature <- paste(feature, as.character(i), sep="")
#dataX_expanded[new.feature] <- with(dataX_expanded,
# ifelse(eval(parse(text=feature)) < i/slope+xmin & eval(parse(text=feature)) > (i-1)/slope+xmin,
# eval(parse(text=feature)) - xmin - eval(parse(text=feature.bin))/slope,
# 0.0
# )
#)
#dataX_expanded[new.feature] <- with(dataX_expanded,
# ifelse(eval(parse(text=feature)) < i/slope+xmin & eval(parse(text=feature)) > (i-1)/slope+xmin,
# eval(parse(text=feature)),
# 0.0
# )
#)
dataX.expanded[new.feature] <- with(dataX.expanded,
ifelse(eval(parse(text=feature)) < i/slope+xmin & eval(parse(text=feature)) > (i-1)/slope+xmin,
1,
0.0
)
)
}
}
dataX.expanded
We then remove all the extra columns and just keep x1, x2, …, x10 for each feature x.
dataX.binned <- dataX.expanded[,grepl(regex("\\d$"),names(dataX.expanded))]
dataX.binned
We also need to remove the columns that contain only zeroes, because they cause problems in the application of the model.
means <- sapply(dataX.binned, mean)
dataX.final <- dataX.binned[which(means > 0)]
dataX.final
Now we are ready to fit the data with the plsRglm model.
set.seed(123)
model.pls.expanded <- plsRglm(dataY,dataX.final,nt=10,dataPredictY=dataX.final,modele="pls-glm-polr")
____************************************************____
Model: pls-glm-polr
Method: logistic
____Component____ 1 ____
glm.fit: fitted probabilities numerically 0 or 1 occurred
____Component____ 2 ____
glm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurred
____Component____ 3 ____
glm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurred
____Component____ 4 ____
glm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurred
____Component____ 5 ____
glm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurred
____Component____ 6 ____
glm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurred
____Component____ 7 ____
glm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurred
____Component____ 8 ____
glm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurred
____Component____ 9 ____
glm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurredglm.fit: fitted probabilities numerically 0 or 1 occurred
____Component____ 10 ____
____Predicting X without NA neither in X nor in Y____
****________________________________________________****
model.pls.expanded
Number of required components:
[1] 10
Number of successfully computed components:
[1] 10
Coefficients:
[,1]
3|4 -6.494061e+00
4|5 -4.233340e+00
5|6 -9.699601e-01
6|7 1.770676e+00
7|8 4.191299e+00
8|9 7.985257e+00
fixed.acidity1 -5.182182e-01
fixed.acidity2 -1.129911e-01
fixed.acidity3 -5.914274e-02
fixed.acidity4 -1.154156e-01
fixed.acidity5 2.862123e-01
fixed.acidity6 3.389757e-01
fixed.acidity7 1.435659e-01
fixed.acidity8 -1.727532e-01
fixed.acidity9 -4.097428e-02
fixed.acidity10 9.025562e-03
fixed.acidity11 5.920507e-02
fixed.acidity12 -4.415641e-02
fixed.acidity13 -7.125334e-01
fixed.acidity14 -5.827418e-01
fixed.acidity15 -2.832894e-01
fixed.acidity16 1.186239e+00
fixed.acidity17 -1.863217e+00
fixed.acidity18 1.805096e+00
fixed.acidity20 -1.970031e+00
volatile.acidity1 4.077460e+00
volatile.acidity2 2.587004e+00
volatile.acidity3 1.333171e+00
volatile.acidity4 1.165663e+00
volatile.acidity5 7.331239e-01
volatile.acidity6 4.145725e-01
volatile.acidity7 1.082460e-01
volatile.acidity8 1.034790e-01
volatile.acidity9 -1.702506e-01
volatile.acidity10 -4.364826e-01
volatile.acidity11 -7.016081e-01
volatile.acidity12 -1.027963e+00
volatile.acidity13 -1.169550e+00
volatile.acidity14 -8.724708e-01
volatile.acidity15 -1.264333e+00
volatile.acidity16 -1.398794e+00
volatile.acidity17 -1.969510e+00
volatile.acidity18 -2.091438e+00
volatile.acidity19 -4.466069e-01
volatile.acidity20 -1.052737e+00
citric.acid1 1.780439e-01
citric.acid2 1.269171e-01
citric.acid3 2.124994e-01
citric.acid4 2.694033e-01
citric.acid5 2.257149e-01
citric.acid6 -1.078736e-01
citric.acid7 -2.483120e-01
citric.acid8 -2.403205e-02
citric.acid9 1.009071e-01
citric.acid10 -3.648438e-03
citric.acid11 3.838162e-01
citric.acid12 3.640878e-01
citric.acid13 7.298252e-01
citric.acid14 1.808723e+00
citric.acid15 -1.419474e-01
citric.acid17 2.651186e+00
citric.acid20 1.931306e+00
residual.sugar1 3.343470e+00
residual.sugar2 1.616747e+00
residual.sugar3 9.892573e-01
residual.sugar4 4.956498e-01
residual.sugar5 2.493510e-01
residual.sugar6 9.405254e-02
residual.sugar7 -3.458010e-02
residual.sugar8 -2.804988e-02
residual.sugar9 6.852988e-02
residual.sugar10 -1.227755e-01
residual.sugar11 -1.061652e-01
residual.sugar12 -1.719673e-02
residual.sugar13 -1.064788e-01
residual.sugar14 -1.556961e-01
residual.sugar15 -2.633151e-01
residual.sugar16 -7.877038e-01
residual.sugar17 -1.585952e-01
residual.sugar18 3.253778e-01
residual.sugar19 6.477780e-01
residual.sugar20 -2.163893e+00
chlorides1 2.378185e-01
chlorides2 2.100432e-01
chlorides3 1.717226e-02
chlorides4 3.787419e-01
chlorides5 1.733989e-01
chlorides6 1.343740e-01
chlorides7 1.763120e-01
chlorides8 1.171058e-02
chlorides9 -3.502257e-01
chlorides10 -3.088657e-01
chlorides11 -4.479072e-01
chlorides12 -1.561645e-01
chlorides13 -7.557955e-01
chlorides14 -3.708093e-01
chlorides15 -1.045747e-02
chlorides16 -7.679959e-01
chlorides17 -1.030835e+00
chlorides20 3.114531e+00
total.sulfur.dioxide1 -4.189395e-03
total.sulfur.dioxide2 -2.005031e-01
total.sulfur.dioxide3 -4.664131e-03
total.sulfur.dioxide4 -2.968508e-01
total.sulfur.dioxide5 -2.598868e-02
total.sulfur.dioxide6 4.671650e-01
total.sulfur.dioxide7 4.190414e-01
total.sulfur.dioxide8 -1.329759e-01
total.sulfur.dioxide9 -3.342838e-03
total.sulfur.dioxide10 -1.309128e-01
total.sulfur.dioxide11 -1.776151e-01
total.sulfur.dioxide12 1.289858e-01
total.sulfur.dioxide13 7.205884e-02
total.sulfur.dioxide14 1.278677e-01
total.sulfur.dioxide15 1.939871e-01
total.sulfur.dioxide16 -5.832429e-01
total.sulfur.dioxide17 4.969586e+00
total.sulfur.dioxide18 4.600023e+00
total.sulfur.dioxide20 5.479488e+03
pH1 6.653837e-01
pH2 1.453287e+00
pH3 3.752981e-01
pH4 -1.238611e-01
pH5 1.547272e-01
pH6 2.963736e-02
pH7 1.409576e-03
pH8 6.377656e-02
pH9 -2.335318e-02
pH10 -1.709131e-01
pH11 -2.205734e-01
pH12 -5.832958e-02
pH13 -4.672400e-03
pH14 4.915395e-01
pH15 8.555796e-01
pH16 1.377559e+00
pH17 -5.272278e-01
pH18 2.552647e+00
pH20 2.115081e-01
sulphates1 -6.005279e-02
sulphates2 -3.320751e-01
sulphates3 -6.857051e-01
sulphates4 -7.132652e-01
sulphates5 -3.916924e-01
sulphates6 -2.718033e-01
sulphates7 5.937014e-02
sulphates8 1.392082e-01
sulphates9 1.607403e-01
sulphates10 2.225042e-01
sulphates11 2.510526e-01
sulphates12 5.486850e-01
sulphates13 -2.590564e-02
sulphates14 1.392501e-01
sulphates15 3.992617e-01
sulphates16 9.121835e-01
sulphates17 -1.764444e+00
sulphates18 -6.098982e-01
sulphates19 1.172595e+00
sulphates20 7.964866e-01
alcohol1 3.844442e+00
alcohol2 1.852312e+00
alcohol3 7.493680e-01
alcohol4 9.217644e-01
alcohol5 1.093438e+00
alcohol6 6.254462e-01
alcohol7 3.903187e-01
alcohol8 1.513303e-01
alcohol9 -3.627450e-01
alcohol10 -4.906415e-01
alcohol11 -1.021437e+00
alcohol12 -1.334117e+00
alcohol13 -1.551486e+00
alcohol14 -2.112469e+00
alcohol15 -2.336856e+00
alcohol16 -2.350504e+00
alcohol17 -2.429983e+00
alcohol18 -2.754132e+00
alcohol20 3.143425e+00
color1 -3.104816e-01
color2 3.104841e-01
Information criteria and Fit statistics:
AIC BIC Missclassed Chi2_Pearson_Y
Nb_Comp_0 16561.19 16601.86 3661 8388.609
Nb_Comp_1 14709.56 14757.02 3165 6314.419
Nb_Comp_2 14168.40 14222.63 3058 5836.756
Nb_Comp_3 13914.28 13975.29 2968 5610.210
Nb_Comp_4 13806.21 13874.00 2935 5527.397
Nb_Comp_5 13771.44 13846.01 2926 5493.560
Nb_Comp_6 13756.38 13837.73 2924 5477.910
Nb_Comp_7 13751.86 13839.98 2909 5471.762
Nb_Comp_8 13749.86 13844.77 2914 5464.127
Nb_Comp_9 13751.49 13853.17 2902 5461.685
Nb_Comp_10 13752.72 13861.18 2910 5464.299
Next, we evaluate the model’s performance like before.
predictions <- as.numeric(predict(model_pls_expanded$FinalModel))
observations <- as.numeric(dataY)
table(as.numeric(unlist(dataY)), predictions )
predictions
1 2 3 4 5 6
1 1 4 16 9 0 0
2 0 0 143 69 4 0
3 0 6 1309 807 16 0
4 0 0 622 1994 218 1
5 0 0 33 765 279 2
6 0 0 1 122 67 3
7 0 0 0 1 4 0
plot(observations, predictions)
mae(observations, predictions)
[1] 0.4927648
This is the best MAE that I’ve seen so far, but I don’t think it’s good enough to justify the amount of effort we put into it and the long time it took to run it. It will also be difficult to expand the model to incorporate new data, since we removed many columns of zeros before fitting.
In conclusion, I would say that the most accurate and robust method that I have found is that of multiplying columns together to effectively turn the plsRglm model into a polynomial model. This method performed better than a simple plsRglm model and can easily be applied to new data. It could probably be improved further by engineering cubic and quartic terms, but the number of features would increase correspondingly and it would take much longer to run. I implemented the quadratic model in the script AFS_wine_quality_model_function.R. At this point, no ensemble method is applied.